1. Field of the Invention
The present invention relates to a dynamic range extending technique for extending a tone range by compositing a plurality of differently exposed images.
2. Description of the Related Art
In recent years, some of image capturing apparatuses such as digital cameras and digital video cameras have a dynamic range extending function which obtains an image whose tone range is extended by compositing a plurality of differently exposed images, which are obtained by capturing images of an object using different exposure times.
A basic concept of a dynamic range extending technique, that is, a so-called HDR (High Dynamic Range) image generation technique, used in the dynamic range extending function is as follows.                Image elements of a blown-out highlight region in an overexposed image are compensated for from those of images captured to have shorter exposure times than that image.        Image elements of a shadow-detail loss region in an underexposed image are compensated for from those of images captured to have longer exposure times than that image.        
In this manner, the tone range of an image can be extended without causing any shadow-detail loss or blown-out highlight. More specifically, respective image elements of an image (HDR image) to be generated, the tone range of which is extended, are generated by executing processing such as replacement to image elements of differently exposed images or a properly exposed image, or weighted averaging of all images used in the processing. Note that differently exposed images are composited by the weighted averaging processing or the like after they are normalized since these images have different tone levels caused by different exposure times.
Generation of an HDR image by such dynamic range extending technique requires a plurality of images to be composited, which are obtained by a plurality of image capturing operations having different exposure levels. However, when objects include a moving object, images to be composited have different moving object regions, an image discontinuity problem occurs in an HDR image obtained by compositing the images to be composited.
More specifically, when object include a moving object, an HDR image obtained by compositing images obtained by a plurality of image capturing operations suffers the following problems associated with image discontinuity. For example, when a case will be assumed wherein a moving object region 201 includes a blown-out highlight region 202 in an overexposed image 200, as shown in FIG. 2, the following problems are posed.    1. When image elements are replaced using those of an underexposed image 220 including no moving object in a region at the same position as the blown-out highlight region 202, a transparent region 231 where a background is seen through is generated in the moving object region of an HDR image 230 (transparency).    2. When image elements are replaced using those of an underexposed image 220 in which a moving object exists in a region at the same position as the blown-out highlight region 202 but its position is different, a texture pattern of a moving object surface becomes discontinuous at a boundary of a corresponding region 232.    3. Unlike in FIG. 2, when an overexposed image includes a blown-out highlight region in a background region, and when a moving image in an underexposed image exists at an occluding position of that region, images are composited by replacement of image elements of the blown-out highlight region to those of the underexposed image as if there were a plurality of images of the moving object (image lag).
Not only image element replacement processing but also composition processing using weighted averaging of a plurality of images in a halftone range also cause image discontinuity such as multiple edges around a moving object and pattern mixture in added moving object regions as problems posed at the time of the conventional image composition processing.
In order to avoid image discontinuity in a composite image output by the HDR image generation processing when a moving object is included, techniques for dynamically controlling image composition in the HDR image generation processing have been disclosed.
Japanese Patent Laid-Open No. 2002-101347 discloses the following technique. That is, when an HDR image is generated using two images, that is, an underexposed image and overexposed image, a region where image discontinuity may occur is specified based on differences between tone levels of normalized images, and replacement to the underexposed image is inhibited in that region.
Japanese Patent Laid-Open No. 10-243288 discloses the following technique. That is, in composition processing by means of weighted averaging of a plurality of images in a halftone range, control is made not to use an image in which motion vectors of a moving object between images to be composited are equal to or larger than a threshold in the composition processing, thus avoiding multiple edges and pattern mixture.
However, the method of Japanese Patent Laid-Open No. 2002-101347 above has been proposed under the assumption of a case in which an image of a region (blown-out highlight region) having tone levels equal to or larger than a threshold in the overexposed image is replaced by the underexposed image, and does not consider any case in which the underexposed image includes a shadow-detail loss region. That is, the aforementioned technique of Japanese Patent Laid-Open No. 2002-101347 eliminates image discontinuity in a blown-out highlight region, but it cannot simultaneously eliminate image discontinuity in all of a blown-out highlight region, halftone region, and shadow-detail loss region.
Since the method of Japanese Patent Laid-Open No. 2002-101347 assumes only the case in which two differently exposed images are to be composited, it is not applicable to composition of three or more images.
Furthermore, the method of Japanese Patent Laid-Open No. 10-243288 avoids image discontinuity in a halftone region. However, since this method requires hardware or processes for detecting motion vectors, other problems about an increase in circuit scale, that in processing time associated with calculations, and so forth may be posed.